1. Technical Field
The invention generally relates to the area of human-computer interaction, and more particularly relates to techniques for selecting objects being displayed or controlling motion and configuration of a virtual object being displayed. With one embodiment of the present invention, inputs from a hand-held controller containing inertial sensors allow a user to control an object within a two or three dimensional representation shown to the user, and allow the user to directly manipulate one or more chosen objects by mapping their location and orientation in a virtual space to those of the user in a physical world. Various embodiments of the present invention may be used in computer applications, video games, or on-screen controls for electronic devices.
2. Related Art
There are a number of man-machine interface devices, such as computer mice, joysticks, remote controllers and trackballs, for controlling computer applications and video games. Each of these devices are well understood in the art and primarily focus on converting motions of a human being into an analog motion that is represented on a two dimensional screen. For example, a joystick translates the position of the control stick relative to a center into a velocity that is applied to a point located on a two dimension screen. According to an established convention, a left-right motion of the control stick corresponds to left-right motion on the screen and a forward-away motion of the control stick corresponds to up-down motion on the screen.
This basic approach of remapping inputs to control motions has been extended to cover three-dimensional computer applications using objects such as 3D mice and 3D joysticks. One approach of doing this is described in U.S. Pat. No. 5,898,421. Most of these approaches available today, however, have the disadvantage that the users must learn an artificial convention for how their motions in the physical world correspond to the motions of a pointer in a computer representation. In general, users prefer natural interactions with a computer application.
A natural interaction for a user would be to have a direct control over the motion of an object in a displayed scene. For example, in a sword-fighting game, a natural control for the user would be to have the sword displayed in the game with the same orientation and position as the motion controller in his/her hand. Currently this is possible by having an external system that measures the exact position and/or orientation of the controller in the physical world. A system for doing this is described in U.S. Pat. No. 4,862,152 but requires the addition of bulky sonic sensors and emitters in the vicinity of the user. Essentially, the system is limited by restricting the motions of the user within a predefined range.
Another natural interaction users desire is the ability to directly point at objects by using their hand to point at the image shown on the display. A two-dimensional solution to this particular style of interaction is introduced by Nintendo in the Wii system (US Patent Publication No.: US20070060384), however it requires additional modification of the environment of the user by adding a sensor bar to define a limited range and field of view, restricting the movements of the user to a small range in front of the sensor bar. It would be desirable to have an approach that requires less modification to the user's environment and allows natural three-dimensional pointing interactions.
There is thus a need for techniques that facilitate full control on motions of displayed objects in positions and orientations in six degrees of freedom. Such techniques shall also work in situations in which there are no additional sensors or emitters, or a motion controller is not able to be detected by some or all of the sensors. There is another need for techniques that provide the ability for users to directly select or point at a portion of a displayed virtual environment in 3D, where the portion of the displayed virtual environment may be an object or a part of a scene in the virtual environment.